Image processing apparatus

ABSTRACT

An apparatus that embeds data in an image includes a mark embedding unit that embeds a feature that forms a predetermined pattern in the image, as a mark to be used to specify an area in which the data is embedded, and a data embedding unit that embeds the data in the image of the area specified by the mark.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technology for embedding data into animage and extracting embedded data from a printed image.

2. Description of the Related Art

Recently, an electronic watermark technique of embedding invisible datainto image data has been actively developed. For example, JapanesePatent Application Laid-Open No. 2004-349879 discloses a technique ofdividing image data into plural blocks, and embedding plural codes intothe blocks by relating one code to each block based on a magnituderelation of characteristics of the block such as average concentrationof the block.

Japanese Patent Application Laid-Open No. H11-187245 discloses atechnique of scanning a total image using a filter of a constant size,embedding data at plural positions where a predetermined condition issatisfied, and extracting data by scanning the image using the samefilter, thereby detecting the positions where the predeterminedcondition is satisfied, and extracting the data.

However, according to the conventional techniques, at a time of readinga printed image to extract data, the data cannot be extracted correctlydue to a distortion of the image, such as an enlargement or reduction ora rotation of the image, at a time of reading the image.

FIG. 19 is a schematic for illustrating a problem occurring when animage is reduced and rotated. In FIG. 19, y denotes a pixel of an areain which data is embedded, and x denotes a pixel of another area. Whenthe image is reduced and rotated, the area embedded with the data doesnot coincide with a filtered area at the time of extracting the data.Therefore, the data cannot be correctly extracted.

SUMMARY OF THE INVENTION

It is an object of the present invention to at least solve the problemsin the conventional technology.

An apparatus that embeds data in an image, according to one aspect ofthe present invention, includes a mark embedding unit that embeds afeature that forms a predetermined pattern in the image, as a mark to beused to specify an area in which the data is embedded; and a dataembedding unit that embeds the data in the image of the area specifiedby the mark.

A method of embedding data into an image, according to another aspect ofthe present invention, includes embedding a feature that forms apredetermined pattern in the image, as a mark to be used to specify anarea in which the data is embedded; and embedding the data in the imageof the area specified by the mark.

A computer-readable recording medium according to still another aspectof the present invention stores a computer program for embedding data inan image. The computer program causes a computer to execute embedding afeature that forms a predetermined pattern in the image, as a mark to beused to specify an area in which the data is embedded; and embedding thedata in the image of the area specified by the mark.

An apparatus that extracts data embedded in an image, according to stillanother aspect of the present invention, includes a mark detecting unitthat detects a mark embedded in the image as a predetermined pattern tospecify an area in which the data is embedded; and a data extractingunit that extracts the data from the area specified based on the mark.

A method of extracting data embedded in an image, according to stillanother aspect of the present invention, includes detecting a markembedded in the image as a predetermined pattern to specify an area inwhich the data is embedded; and extracting the data from the areaspecified based on the mark.

A computer-readable recording medium according to still another aspectof the present invention stores a computer program for extracting dataembedded in an image. The computer program causes a computer to executedetecting a mark embedded in the image as a predetermined pattern tospecify an area in which the data is embedded; and extracting the datafrom the area specified based on the mark.

A printed medium according to still another aspect of the presentinvention has an image data and a mark printed thereon. The image datais divided into a plurality of blocks. A code is embedded in an area ofthe image data based on a magnitude relation of a characteristic amountof each of the blocks between a pair of blocks. The mark is a featureforming a predetermined pattern in an image, to be used to specify thearea in which the code is embedded area.

The above and other objects, features, advantages and technical andindustrial significance of this invention will be better understood byreading the following detailed description of presently preferredembodiments of the invention, when considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic for illustrating a method of specifying adata-embedded area by an image processing apparatus according to anembodiment of the present invention;

FIG. 2 is a functional block diagram of an encoder and a decoderaccording to the present embodiment;

FIG. 3 is a schematic for illustrating a margin between a mark and adata-embedded area;

FIG. 4 is a schematic for illustrating an area in which marks are to beembedded;

FIG. 5 is a schematic for illustrating an outline of a mark embeddingprocess performed by a mark embedding unit;

FIGS. 6A to 6D are schematics for illustrating an inflection point;

FIG. 7 is a schematic for illustrating embedding of marks as asymmetrical pattern of the inflection point;

FIGS. 8A and 8B are schematics for illustrating other symmetricalpatterns;

FIG. 9 is a flowchart of a processing procedure performed by a markembedding unit;

FIGS. 10A to 10C are schematics for illustrating mark-embeddingpositions according to shapes of data-embedded areas;

FIG. 11 is a schematic for illustrating a wavelet transform;

FIG. 12 is a flowchart of a processing procedure for aninflection-point-pattern embedding process performed by aninflection-point-pattern embedding unit;

FIG. 13 is a schematic for illustrating a method of detecting aninflection point;

FIGS. 14A and 14B are schematics for illustrating a method of selectingan embedding pattern;

FIG. 15 is a flowchart of a processing procedure for a value addingprocess procedure;

FIG. 16 is a diagram for explaining an example of creating positive andnegative inflection points in a specific frequency component by addingvalues;

FIG. 17 is a flowchart of a processing procedure for a dataembedded-area detecting process performed by an area detecting unit;

FIG. 18 is a functional block diagram of a computer that executes anencoding program according to the present embodiment; and

FIG. 19 is a schematic for illustrating a problem occurring when animage is reduced and rotated.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Exemplary embodiments of the present invention are explained in detailbelow with reference to the accompanying drawings.

Throughout the specification, an image processing apparatus that embedsdata into an image is referred to as an encoder, and an image processingapparatus that extracts embedded data from an image is referred to as adecoder.

FIG. 1 is a schematic for illustrating a method of specifying adata-embedded area by an image processing apparatus according to anembodiment of the present invention. An encoder embeds invisible marksinto four areas that encircle a data-embedded area. The encoder embedsmarks by changing a tone level of one of R, G, and B planes so that aresult of transforming a wavelet of the tone level becomes apredetermined pattern of an inflection point.

At the time of extracting data, a decoder detects four marks from theread image, and detects an enlargement or reduction rate and a rotationangle of the data-embedded area from a positional relationship of thesemarks. The decoder specifies the data-embedded area by using thedetected enlargement or reduction rate and the detected rotation angle.

According to the present embodiment, the encoder embeds marks forspecifying a data-embedded area into an image. The decoder detects themarks embedded in the image, and detects an enlargement or reductionrate and a rotation angle of the read image. With this mechanism, it ispossible to accurately specify the data-embedded area, and extract theembedded data with high precision.

FIG. 2 is a functional block diagram of an encoder 100 and a decoder 200according to the present embodiment. The encoder 100 includes animage-data memory unit 110, an image-data storing unit 120, a markembedding unit 130, a data embedding unit 140, and an image printingunit 150. The decoder 200 includes an image input unit 210, animage-data memory unit 220, an area detecting unit 230, and a decodeprocessing unit 240.

The image-data memory unit 110 stores data of an image in which the datais embedded. Specifically, the image-data memory unit 110 stores tonelevels of R, G, and B for each pixel, and also stores coordinates atwhich marks are embedded.

The image-data storing unit 120 reads an image from an imaging device,and stores the image data into the image-data memory unit 110. Theimage-data storing unit 120 can also read image data from an image datafile and store the image data into the image-data memory unit 110.

The mark embedding unit 130 embeds marks for specifying a data-embeddedarea into image data. The mark embedding unit 130 includes a frequencyconverting unit 131 that frequency-converts a tone level of one of R, G,and B planes, and an inflection-point-pattern embedding unit 132 thatchanges the tone level so that a frequency-converted result becomes apredetermined pattern of the inflection point.

The data embedding unit 140 embeds data into a data-embedding areaspecified by marks embedded by the mark embedding unit 130. The dataembedding unit 140 reads image data from the image-data memory unit 110,embeds data into the read image data, and stores the embedded resultinto the image-data memory unit 110.

The image printing unit 150 prints an image which is embedded with data.The image printing unit 150 reads data-embedded image data from theimage-data memory unit 110, and prints the image data.

The image input unit 210 picks up a printed image with an imagingdevice, and inputs the image. The image input unit 210 stores the imagedata of the input image into the image-data memory unit 220. Theimage-data memory unit 220 stores the image data of the image input bythe image input unit 210.

The area detecting unit 230 detects a data-embedded area. The areadetecting unit 230 detects marks embedded into an image by the encoder100, calculates an enlargement or reduction rate and a rotation angle ofthe input image using the detected marks, and detects the data-embeddedarea.

FIG. 3 is a schematic for illustrating a margin between a mark and adata-embedded area. The mark embedding unit 130 leaves a constant marginbetween each mark and the data-embedding area, at the time of embeddingthe marks at corners of a square data-embedded area.

In other words, the mark embedding unit 130 creates an area (a quietzone) in which no data is included between each mark and thedata-embedded area. By providing the quiet zone, the marks can bedetected more accurately.

FIG. 4 is a schematic for illustrating an area in which marks are to beembedded. The area in which the marks are to be embedded is divided intoplural blocks. Each block includes plural pixels. In FIG. 4, the totalarea in which the marks are to be embedded (n1×n2 pixels) is dividedinto blocks of 3×3. Each block includes m1×m2 pixels. As shown in FIG.4, the area in which the marks are to be embedded has the layout ofblocks each having plural pixels.

FIG. 5 is a schematic for illustrating an outline of a mark embeddingprocess performed by the mark embedding unit 130. The mark embeddingunit 130 wavelet transforms a tone level of one of R, G, and B planesfrom the data of the image into which data is to be embedded, andchanges the tone level corresponding to the mark-embedding area of an LLpart so that the tone level becomes a predetermined pattern.

For example, assume that a pixel having a tone level smaller than athreshold value is expressed as “0”, and a pixel having a tone levelequal to or larger than the threshold value is expressed as “1”. When apattern “01010” is obtained based on this assumption, this pattern isassumed as a pattern of the mark. When an actual binary result has apattern “01000”, the mark embedding unit 130 increases a tone level ofthe fourth pixel “0” to “1”. Similarly, the mark embedding unit 130 alsochanges a pixel tone level of a pattern in other rows so that thepattern of the mark is obtained. With this arrangement, the markembedding unit 130 can embed the pattern of the mark into the tonelevels of the pixel. In FIG. 5, a pattern of five rows and five columns

0 0 0 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 0 0 0 0 0 0is embedded as the mark.

These patterns have a characteristic in that the pattern “01010” can beobtained even when the patterns are scanned from any direction, and thatthese patterns are rotationally symmetrical. When rotation-symmetricalpatterns are used as marks, these marks can be specified accurately evenwhen the image is rotated. When patterns of five rows and ten columns

0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 1 1 1 11 1 0 0 0 0 0 0 0 0 0 0 0 0are used, the pattern “01010” can be accurately specified even when theimage is reduced in a lateral direction.

For the sake of convenience, structuring of patterns based on amagnitude relation between a tone level of each pixel and a thresholdvalue has been explained. However, in actual practice, patterns areformed based on an inflection point of tone levels of plural pixels thatconstitute the blocks. A pattern formed based on an inflection point isexplained next.

FIGS. 6A to 6D are schematics for illustrating an inflection point.FIGS. 6A and 6B show a negative inflection point having an inflectionpoint projected upward, and FIGS. 6C and 6D show a positive inflectionpoint having an inflection point projected downward.

FIG. 7 is a schematic for illustrating embedding of marks as asymmetrical pattern of the inflection point. When a negative inflectionpoint is black and when a positive inflection point is white, the markembedding unit 130 embeds a symmetrical pattern of white and black asmarks.

The mark embedding unit 130 changes tone levels of pixels according toneed, and creates a symmetrical pattern of inflection points as shown inFIG. 7, thereby embedding marks into the image. When a symmetricalpattern to be embedded as marks is symmetrical about a point, othersymmetrical patters can be also used.

FIGS. 8A and 8B are schematics for illustrating other symmetricalpatterns. FIG. 8A shows the pattern shown in FIG. 7 of which positiveand negative and black and white points are inverted. FIG. 8B shows aconcentric pattern. The white and black patterns shown in FIG. 7 andFIGS. 8A and 8B correspond to the patterns of “0” and “1” shown in FIG.5.

FIG. 9 is a flowchart of a processing procedure performed by the markembedding unit 130. The mark embedding unit 130 reads mark-embeddingcoordinates from the image-data memory unit 110 (step S101).

When a data-embedded area does not have a square shape, mark-embeddingpositions can be determined based on the shape of the data-embeddedarea. FIGS. 10A to 10C depict mark-embedding positions according toshapes of data-embedded areas.

As shown in FIG. 10A, when a shape of a data-embedded area is triangle,four crests of a rectangle that encircles this triangle are set aspositions into which marks are to be embedded. As shown in FIG. 10B,when a shape of a data-embedded area is irregular, four crests of arectangle that encircles this area are set as positions into which marksare to be embedded. As shown in FIG. 10C, when a shape of adata-embedded area is circle, crests of a triangle that encircles thiscircle are set as positions into which marks are to be embedded.

The frequency converting unit 131 frequency-converts the tone level(step S102), thereby extracting a band in which embedding is performed.The frequency converting unit 131 performs a wavelet transform as afrequency conversion process.

FIG. 11 is a schematic for illustrating a wavelet transform. The wavelettransform is performed by dividing an image into four band components of1LL, 1LH, 1HL, and 1HH. 1LL indicates that wavelet transform isperformed once, and includes an L (low frequency) component in the xdirection, and an L component in the y direction. 1HH indicates thatwavelet transform is performed once, and includes an H (high frequency)component in the x direction, and an H component in the y direction.

A second wavelet transform is performed for the 1LL components. Thecomponents of the 1LL are divided into four bands of 2LL, 2LH, 2HL, and2HH. The wavelet transform is performed plural times until when a bandcomponent in which the embedding process is performed is extracted.

While the wavelet transform is used for the frequency conversionprocess, a band component in which the embedding process is performedcan be extracted by executing other process such as a moving average.

The inflection-point-pattern embedding unit 132 embeds markscorresponding to the extracted band component (step S103). The frequencyconversion process is performed to check a direction and strength of theinflection point of the area to which the marks are added. The markembedding process to add values is performed to the original image. Inother words, values are not added to frequency-resolved component.Therefore, the frequency conversion process can be an irreversibleconversion process.

The frequency converting unit 131 converts the frequency of the tonelevel, thereby extracting the mark-embedded band. Theinflection-point-pattern embedding unit 132 embeds marks into the bandcomponent extracted by the frequency converting unit 131 so that apattern of the inflection point can be formed. With this arrangement,the mark embedding unit 130 can embed marks with minimum imagedegradation.

FIG. 12 is a flowchart of a processing procedure for aninflection-point-pattern embedding process performed by theinflection-point-pattern embedding unit 132.

In the inflection-point-pattern embedding process, theinflection-point-pattern embedding unit 132 initializes mark_count to“0” (step S201). The mark_count represents a variable to count thenumber of processed marks.

It is determined whether mark_count is smaller than the number ofembedded marks, mark_num (step S202). When mark_count is not smallerthan mark_num, it means that all marks are embedded, therefore, theprocess ends.

On the other hand, when mark_count is smaller than mark_num, the areainto which marks are embedded is divided into blocks (step S203). Theinfection point of each block is detected, and an embedding pattern isselected (step S204).

FIG. 13 is a schematic for illustrating a method of detecting aninflection point. The inflection-point-pattern embedding unit 132calculates differences between the tone level of a pixel at the centerof the block and the tone level of each pixel at both ends of the blockrespectively, and detects an inflection point based on positive ornegative of a total of these differences. For example, differencesbetween the tone level of a pixel at the center of the block and thetone level of each pixel at both ends of the block respectively are “10”and “15”, and the total of these differences is “25”, being positive.Therefore, an inflection point projected upward is detected. In FIG.13B, differences between the tone level of a pixel at the center of theblock and the tone level of each pixel at both ends of the blockrespectively are “20” and “−5”, and the total of these differences is“15”, being positive. Therefore, an inflection point projected upward isdetected.

An inflection point can be also detected by carrying out differentiationtwice within the area of the block. In other words, when adifferentiation value is negative twice, the inflection point isprojected upward, and when a differentiation value is positive twice,the inflection point is projected downward.

FIGS. 14A and 14B are schematics for illustrating a method of selectingan embedding pattern. The inflection-point-pattern embedding unit 132compares a result of calculating inflection points with a pattern, andselects a pattern that has a larger number of bocks of which inflectionpoints coincide with the pattern, as an embedding pattern. For example,in FIG. 14A, the number of coincidence between the result of calculatinginflection points and a “pattern 1” is “7”, and the number ofcoincidence between the result of calculating inflection points and a“pattern 2” is “2”. Therefore, the inflection-point-pattern embeddingunit 132 selects the “pattern 1” as an embedding pattern.

The inflection-point-pattern embedding unit 132 initializes the numberof variables block_count to count the number of processed blocks (stepS205), and determines whether the number of variables block_count issmaller than the number of blocks, block_num (step S206).

As a result, when block_count is smaller than block_num, theinflection-point-pattern embedding unit 132 executes a value addingprocess to a block that does not coincide with the pattern, to changepositive or negative of inflection points (step S207). Theinflection-point-pattern embedding unit 132 adds “1” to block_count(step S208), and the process returns to step S206.

On the other hand, when block_count is not smaller than block_num, itmeans that the process of all blocks have ended, and therefore, “1” isadded to mark_count (step S209), and the process returns to step S202.

The inflection-point-pattern embedding unit 132 divides themark-embedded area into blocks, and changes a tone level of a pixel sothat a pattern formed by the inflection points of blocks becomes apredetermined symmetric pattern. With this arrangement, marks can beembedded as a pattern of inflection points.

FIG. 15 is a flowchart of a processing procedure for a value addingprocess procedure. An embedding level is calculated in the value addingprocess (step S301). The embedding level is a total of the differencesshown in FIG. 13. When detecting inflection points by carrying outdifferentiation twice in the block area, a differential value of twiceat the center becomes the embedding level.

A value is added so that positive and negative of inflection points areinverted exceeding the calculated embedding level (step S302).Specifically, positive and negative of inflection points are inverted byadding a value of a trigonometric function to an original imageaccording to the embedding level.

FIG. 16 is a diagram for explaining an example of creating positive andnegative inflection points in a specific frequency component by addingvalues. The upper graph in FIG. 16 expresses an original image and aspecific frequency component. According to this specific frequencycomponent, inflection points are obtained as positive, negative, andnegative in this order from the left.

Values are added to the original image so that the inflection pointsbecome in the patterns of negative, positive, and negative. To change apositive pattern to a negative pattern at the left side, values areadded so that the strength calculated at step S301 is exceeded. Tochange a negative pattern to a positive pattern, positive values aresimilarly added to the center. Since the right side originally shows anegative inflection point, no value is added to the right side. As aresult, a graph as shown at the lower side of FIG. 16 is obtained. Inthe specific frequency component of the graph shown at the lower side,the inflection points are negative, positive, and negative from the leftin this order.

The inflection-point-pattern embedding unit 132 calculates the embeddinglevel, and adds values to invert positive and negative of inflectionpoints exceeding the calculated embedding level, thereby changing thepositive and negative of the inflection points of blocks.

FIG. 17 is a flowchart of a processing procedure for a dataembedded-area detecting process performed by the area detecting unit230.

The area detecting unit 230 performs the frequency conversion process ina similar manner to that performed by the frequency converting unit 131of the encoder 100 using data of a tone level of a pixel stored in theimage-data memory unit 220 (step S401). The area detecting unit 230detects inflection points and inflection point patterns by executing theprocess similar to that performed by the inflection-point-patternembedding unit 132 of the encoder 100 (steps S402 and S403).

The area detecting unit 230 determines mark positions based on thedetected inflection point patterns (step S404), and detects anenlargement or reduction rate and a rotation angle (step S405).

The area detecting unit 230 of the decoder 200 detects an inflectionpoint pattern using data of an image stored in the image-data memoryunit 220, and detects mark positions, an enlargement or reduction rate,and a rotation angle using the detected inflection point pattern,thereby accurately specifying a data-embedded area.

According to the present embodiment, the frequency converting unit 131of the encoder 100 executes the frequency conversion of the tone level,thereby extracting a mark-embedded band. The inflection-point-patternembedding unit 132 embeds marks of inflection point patterns into bandcomponents extracted by the frequency converting unit 131. With thisarrangement, the area detecting unit 230 of the decoder 200 can detect adata-embedded area, an enlargement or reduction rate, and a rotationangle of the image without affecting the visible image, and the decoder200 can extract data accurately.

Configurations of the encoder-and the decoder can be also realized withsoftware to obtain an encoding program and a decode program havingfunctions similar to those of the encoder and the decoder.

FIG. 18 is a functional block diagram of a computer that executes anencoding program according to the present embodiment. A computer 300 hasa random access memory (RAM) 310., a central processing unit (CPU) 320,a hard disk drive (HDD) 330, a local-area-network (LAN) interface 340,an input and output (I/O) interface 350, and a digital-versatile-disk(DVD) drive 360.

The RAM 310 is a memory that stores a program and an interim result ofexecuting the program. The CPU 320 is a central processing unit thatreads the program from the RAM 310, and executes this program.

The HDD 330 is a disk drive that stores a program and data. The LANinterface 340 connects the computer 300 with other computers via a LAN.

The I/O interface 350 connects input and output devices such as a mouse,a keyboard, a display unit, and a printer. The DVD drive 360 reads fromand writes into a DVD.

An encoding program 311 executed by the computer 300 is stored into theDVD. The DVD drive 360 reads the encoding program 311 from the DVD, andinstalls the encoding program into the computer 300.

Alternatively, the encoding program 311 is stored into a database ofanother computer system connected via the LAN interface 340, and is readfrom the database and is installed in the computer 300.

The encoding program 311 is stored into the HDD 330, and is read intothe RAM 310. The CPU 320 executes this encoding program as an encodingprocess 321.

Although the invention has been described with respect to a specificembodiment for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

1. An apparatus that embeds data in an image, the apparatus comprising:a mark embedding unit that embeds a feature that forms a predeterminedpattern in the image, as a mark to be used to specify an area in whichthe data is embedded; and a data embedding unit that embeds the data inthe image of the area specified by the mark, wherein the mark embeddingunit embeds a pattern of inflection points into a tone level of aspecific color plane, as the mark.
 2. The apparatus according to claim1, wherein the predetermined pattern is symmetrical with respect to apoint at a center of the pattern in the image.
 3. A computer-implementedmethod of embedding data into an image, the method comprising: firstembedding a feature that forms a predetermined pattern in the image, asa mark to be used to specify an area in which the data is embedded; andsecond embedding the data in the image of the area specified by themark, wherein the first embedding embeds a pattern of inflection pointsinto a tone level of a specific color plane, as the mark.
 4. Acomputer-readable recording medium that stores a computer program forembedding data in an image, wherein the computer program causes acomputer to execute: first embedding a feature that forms apredetermined pattern in the image, as a mark to be used to specify anarea in which the data is embedded; and second embedding the data in theimage of the area specified by the mark, wherein the first embeddingembeds a pattern of inflection points into a tone level of a specificcolor plane, as the mark.
 5. An apparatus that extracts data embedded inan image, the apparatus comprising: a mark detecting unit that detects amark embedded in the image as a predetermined pattern to specify an areain which the data is embedded; and a data extracting unit that extractsthe data from the area specified based on the mark, wherein the markdetecting unit detects a mark that is embedded in a tone level of aspecific color plane as a pattern of inflection points.
 6. Acomputer-implemented method of extracting data embedded in an image,comprising: detecting a mark embedded in the image as a predeterminedpattern to specify an area in which the data is embedded; and extractingthe data from the area specified based on the mark, wherein thedetecting detects a mark that is embedded in a tone level of a specificcolor plane as a pattern of inflection points.
 7. A computer-readablerecording medium that stores a computer program for extracting dataembedded in an image, wherein the computer program causes a computer toexecute: detecting a mark embedded in the image as a predeterminedpattern to specify an area in which data is embedded; and extracting thedata from the area specified based on the mark, wherein the detectingdetects a mark that is embedded in a tone level of a specific colorplane as a pattern of inflection points.